Layout Representation Learning with Spatial and Structural Hierarchies

We present a novel hierarchical modeling method for layout representation learning, the core of design documents (e.g., user interface, poster, template). Existing works on often ignore element hierarchies, which is an important facet layouts, and mainly rely spatial bounding boxes feature extraction. This paper proposes Spatial-Structural Hierarchical Auto-Encoder (SSH-AE) that learns by treating hierarchically annotated as tree format. On one side, we model SSH-AE from both (semantic views) structural (organization relationships) perspectives, are two complementary aspects to represent layout. other semantic/geometric properties associated at multiple resolutions/granularities, naturally handling complex layouts. Our learned representations used effective search similarity perspectives. also newly involve tree-edit distance (TED) evaluation metric construct comprehensive protocol assessment, benefits systematic customized search. further new dataset POSTER layouts believe will be useful future research. show our proposed outperforms existing methods achieving state-of-the-art performance benchmark datasets. Code available github.com/yueb17/SSH-AE.